# PubMed Pretraining

BioMistral is an open-source medical domain model suite based on the Mistral architecture, further pretrained using PubMed Central open-access text data.

BioGPT is a domain-specific generative pre-trained Transformer language model trained on large-scale biomedical literature, focusing on biomedical text generation and mining.

Large-scale biomedical language model based on BERT, ALBERT, and ELECTRA, specialized for biomedical domain tasks

A biomedical domain-specific BERT model pretrained on PubMed article abstracts, achieving state-of-the-art performance in multiple biomedical NLP tasks

BioBERT is a BERT-based pretrained language model specifically optimized for biomedical text mining tasks, suitable for question answering systems.